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THE  VARIABILITY  OF  SPATIAL  RESOLUTION  ESTIMATES 
OBTAINED  USING  A  CCD  CAMERA 


SUMMARY 

We  have  previously  developed  techniques  for  measuring  the  spatial  resolution  of  flight 
simulator  displays.  In  the  present  experiment,  we  estimate  the  relative  variability  of  the 
measurement  technique  and  the  display  projectors  they  were  designed  to  assess.  We  use  the 
ratio  of  the  standard  deviation  and  mean  of  the  resolution  estimates  as  a  measure  of  variability. 
Variability  was  found  to  be  about  1.3%  for  grating  transparencies  illuminated  by  a  stable  light 
source.  This  value  may  be  taken  as  the  inherent  variability  of  our  measurement  hardware  and 
analysis  procedures.  Analogous  measurements  made  on  cathode  ray  tube  (CRT)  projectors 
resulted  in  a  mean  variability  of  about  4.3%.  The  difference  between  the  two  estimates,  3.0%, 
may  be  taken  as  a  measure  of  the  variability  of  the  CRT  projectors  alone. 

BACKGROUND  /  INTRODUCTION 

In  many  flight-simulator  applications,  spatial  resolution  is  the  major  determinant  of  the 
fidelity  and  effectiveness  of  displayed  imagery.  Operational  visual  display  systems  are  often 
difficult  to  calibrate  for  both  practical  reasons,  such  as  limited  working  room  and  limited  access 
time,  as  well  as  technical  reasons,  such  as  the  lack  of  standardized  tests  and  a  general  lack  of 
consensus  as  to  which  tests  are  required,  and  what  the  results  of  those  tests  mean.  Further,  it  is 
well  known  that  projector  resolution  decreases  with  age.  In  addition,  variations  in  routine  setup 
procedures,  such  as  focusing  and  color  convergence,  can  alter  display  resolution.  For  these 
reasons,  it  is  good  practice  to  objectively  measure  display  resolution  on  a  regular  basis. 

We  have  previously  developed  a  technique  for  measuring,  specifying,  and  conceptualizing 
the  spatial  resolution  of  visual  displays  (Geri,  Winterbottom,  &  Pierce,  2004).  That  technique  is 
based  on  a  well-established  measurement  standard  (VESA,  1998).  However,  the  precision  of 
that  technique  is  limited  by  the  variability  of  both  the  light  measurement  device  and  the 
procedures  used  to  estimate  spatial  resolution.  Since  very  little  data  are  available  concerning 
either  of  these  sources  of  variability,  we  have  attempted  to  assess  them  here. 

We  performed  three  related  evaluations  of  spatial  resolution  variability:  two  on  CRT- 
projectors,  and  one  using  grating  transparencies  illuminated  by  a  stable  light  source.  The  first 
projector  evaluation  examined  the  variability  of  the  display’s  spatial  resolution  across  a  six- 
hour  test-period  and  over  four  days.  The  second  projector  evaluation  measured  spatial  resolution 
variability  within  each  test-period.  Finally,  the  grating  transparency  evaluation  used  a  stable 
light  source  and  a  set  of  grayscale  gratings  to  distinguish  what  percentage  of  the  variation  was 
due  to  the  CRT  monitor,  as  compared  to  the  measurement  hardware,  and  measurement  and 
analysis  procedures. 


METHOD 

Stimuli  and  Apparatus. 

For  both  the  grating-transparency  and  projector  evaluations,  luminance  measurements  were 
obtained  using  a  CCD  camera  (Model  ST7,  Santa  Barbara  Instrument  Group,  Santa  Barbara, 
CA).  Additional  measurements  were  obtained  using  a  calibrated  Minolta  LS-100  photometer. 


1 


Spatial  resolution  was  estimated  from  the  luminance  distributions  obtained  with  the  CCD 
camera  using  techniques  developed  for  this  purpose  (see  Geri  et  al.,  2004). 


Projector  Evaluations.  The  stimuli  used  to  estimate  projector  resolution  were  generated 
using  special-purpose  software  and  hardware  (Geri  et  al.,  2004).  The  test  images  were 
horizontal  and  vertical  grille  patterns  composed  of  on-off  lines  consisting  of  one,  two,  or  three 
pixels  in  width.  The  lines  of  each  grille  pattern  were  modeled  as  either  full-black  (grayscale  =  0) 
or  full  white  (grayscale  =  255).  The  test  images  were  displayed  using  a  Barco  Model  808  CRT 
projector  (Barco  Inc.,  Kennesaw,  GA)  and  a  Proscreen  Model  1.2  rear-projection  screen 
(Proscreen  Inc.,  Medford  Oregon).  To  match  the  Mobile  Modular  Display  for  Advanced 
Research  and  Training  (M2DART)  settings  used  in  the  operational  simulators  at  the  Air  Force 
Research  Laboratory,  Warfighter  Readiness  Research  Division  (AFRL/HEA),  Mesa  AZ,  the 
dimensions  of  the  rear-projected  image  were  set  to  52  in.  (H)  x  43  in.  (V).  The  distance  from  the 
projector  to  screen  was  97  inches.  The  test  images  were  generated  at  1600  *  1200  pixels  using  a 
PC-based  test  system  that  included  an  AGP  NVIDIA  GForce  4  video  card. 

Grating-Transparency  Evaluation.  Three  transparencies,  consisting  of  the  same  grating 
frequency  at  three  different  contrasts  were  created  using  Illustrator  10  (Adobe,  San  Jose,  CA), 
and  were  printed  at  600  dpi.  The  gratings  consisted  of  alternating  dark  and  light  lines  at  a 
frequency  of  51  lines/in.  The  resulting  line  widths  (0.5  mm)  approximated  those  of  the  1  on/1 
off  grille  pattern  used  in  the  projector  evaluations.  The  nominal  contrast  values  (low,  medium, 
and  high)  were  chosen  to  approximate  those  typically  encountered  when  spatial  resolution 
measurements  are  obtained  from  CRT  projectors.  The  grating  transparencies  were  illuminated 
by  a  stable  incandescent  light  source.  A  1.6  neutral  density  filter  was  used  to  reduce  the  mean 
luminance  of  the  grating  to  about  16  fL,  to  approximately  match  the  mean  luminance  of  the 
imagery  used  in  the  projector  evaluations.  In  addition,  a  heat-absorbing  filter  was  used  to 
remove  near-IR  radiation  to  which  the  CCD  camera  was  sensitive  but  the  Minolta  photometer 
was  not. 

Procedure 

Resolution  measurements  were  obtained  with  the  CCD  camera  placed  16  cm  from  the 
projected  image  under  test.  This  resulted  in  the  CCD-camera  field-of-view  covering  a 
rectangular  area  of  about  7  mm  (H)  x  5mm  on  the  displayed  image. 

For  all  evaluations,  spatial  resolution  estimates  were  calculated  using  an  MS-Excel 
spreadsheet  that  implemented  the  techniques  described  in  Geri  et  al.  (2004).  The  spreadsheet 
routines  located  the  maximum  and  minimum  values  from  each  of  the  measured  grill-pattern 
luminance  distributions,  calculated  their  averages,  and  then  computed  a  Michelson  contrast. 
Horizontal  and  vertical  resolutions  were  obtained  using  a  contrast  criterion  of  25%. 

Grating  Transparency  Evaluation.  The  light  source  that  illuminated  the  gratings  was  first 
allowed  to  warm  up  for  15  min.  The  contrast  of  each  of  the  three  contrast  gratings  was  then 
measured  three  times  for  each  of  three  grating  contrast  levels,  with  the  measurement  order 
randomized.  This  procedure  was  then  repeated  twice,  with  the  camera  moved  and  set  up  again 
between  each  repetition,  resulting  in  3  measurement-sets  within  the  daily  test-period.  The  three 
sets  of  measurements  were  then  repeated  on  each  of  four  test-days.  Spatial  resolution  estimates 
were  obtained  from  each  set  of  measurements  made  on  the  three  (low,  medium,  high)  contrast 
gratings. 
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Projector  Evaluation  1.  The  projector  was  left  on  stand-by  overnight.  At  the  beginning  of 
each  test  day,  the  projector  was  turned  on.  For  the  first  30  min,  the  projector  was  allowed  to 
stabilize  using  Barco’s  built-in  White-Pattern  Warm-Up  Screen  and  then  remained  on  for  the 
remainder  of  the  test-day.  After  this  initial  warm  up  period,  spatial  resolution  measurements 
were  obtained,  and  then  repeated  at  three  hour  and  six  hour  intervals,  for  a  total  of  three  test- 
periods.  Each  of  the  test-periods  lasted  approximately  15  min,  and  all  were  repeated  on  each  of 
four  days. 

The  projector  was  fully  on,  i.e.,  imagery  was  projected,  for  the  entire  test-day.  Before  each 
test-period,  the  projector’s  color  convergence  was  checked  and  adjusted  if  necessary.  For  both 
the  spatial-resolution  and  gamma  measurements  (see  below),  the  camera  and  photometer  were 
moved  and  then  set  up  again  between  each  of  the  three  test-periods. 

Spatial  resolution  estimates  were  obtained  using  horizontal  and  vertical,  1-on/l-off,  2-on/2- 
off  and  3-on/3-off  grille-patterns.  The  order  in  which  the  grille-patterns  were  tested  was 
randomized.  A  gamma  function  was  also  obtained  within  each  test-period  (see  below).  The 
order  in  which  the  grayscale  levels,  used  to  obtain  the  gamma  functions,  were  tested  was  also 
randomized. 

Projector  Evaluation  2.  A  second  projector  evaluation  was  performed  to  assess  the 
variability  of  the  spatial  resolution  estimates  within  each  test-period.  Methods  and  procedures 
were  identical  to  Projector  Evaluation  1  except  that  three  measurements  were  obtained  for  each 
grille  pattern  within  each  test-period.  Each  test-period  lasted  approximately  30  min. 

CCD  Luminance  Calibration.  In  order  to  calibrate  the  data  obtained  from  the  CCD  camera, 
gamma  functions  were  obtained  by  relating  the  CCD  output  to  luminance  measurements 
obtained  using  the  calibrated  Minolta  photometer.  The  order  in  which  the  gamma  function  and 
spatial-resolution  data  were  collected  was  randomized.  For  the  projector  evaluation,  the  gamma 
functions  were  obtained  using  a  series  of  full-screen  images  whose  luminances  corresponded  to 
DAC  values  ranging  from  0  to  255  in  steps  of  30.  The  luminance  of  the  series  of  images  was 
measured  with  both  the  Minolta  photometer  and  the  CCD  camera.  The  output  of  the  latter  varied 
between  0  and  65536  (i.e.,  16-bits).  For  the  gamma  measurements,  the  CCD-camera  and  the 
Minolta  photometer  were  placed  16  cm  and  91  cm,  respectively,  from  the  image  being 
measured.  For  the  grating  transparency  evaluation,  image  luminance  was  varied  using  nine 
neutral  density  filters  ranging  from  1.6  to  4.1  ND.  Each  filter  or  filter  combination  was  placed  in 
front  of  the  heat-absorbing  filter  and  light  source.  For  each  filter,  the  measured  luminance  was 
plotted  as  a  function  of  the  CCD  output,  and  a  power  function  was  fit  to  the  data  using 
SigmaPlot  (v.  8.0,  SPSS  Inc.,  Chicago,  IL). 

RESULTS  AND  DISCUSSION 

Grating-Transparency  Evaluation.  Shown  in  Figure  1  are  mean  contrast  measures 
obtained  for  the  grating  transparencies.  The  results  of  an  Analysis  of  Variance  (ANOVA) 
showed  a  significant  main  effect  of  Test-Day  ,fi,24.M,/?<0.001)  but  not  Measurement-Set  ( 63,24  = 
23.1,  p  =  0.062).  Based  on  the  ANOVA  results,  it  was  decided  to  estimate  CCD-measurement 
variability  by  collapsing  the  data  across  measurement-sets,  and  finding  the  ratio  of  the  standard 
deviation  to  the  mean  of  the  resolution  measures  for  each  test-day.  The  ratios  found  were 
0.0192,  0.0127,  0.00984,  and  0.00828.  The  mean  of  these  four  ratios  is  0.0125  (1.25%),  which 
we  take  as  the  variability  of  our  CCD  measurements  and  spatial-resolution  analysis  procedure. 
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Projector  Evaluation  1.  Shown  in  Figure  2  are  the  horizontal  and  vertical  spatial  resolution 
estimates  obtained  for  the  first  projector  evaluation.  Separate  one-way  ANOVAs  were 
performed  for  the  horizontal  and  vertical  resolution  estimates  for  both  the  Test-Day  and  Test- 
Period  variables.  As  there  was  no  significant  effect  of  either  Test-Day  (.63,8  <  1.0,  p  >  0.44)  or 
Test-Period  (Fs,?<  0.58,  p  >  0.58),  the  horizontal  and  vertical  data  of  Figure  2  were  each  used 
to  estimate  the  variability  of  the  spatial  resolution  measurements  by  the  same  technique  used 
for  the  grating-transparency  data.  The  ratios  of  the  standard  deviation  to  the  mean  of  the  24 
horizontal  and  vertical  measurements  were  found  to  be  0.035  and  0.016,  respectively.  The 
resulting  variability  estimate  for  our  projector  spatial  resolution  measurements  is  therefore 
0.026  (2.6  %). 

It  should  be  noted  that  the  vertical  variability  estimate  obtained  for  the  CRT  projector  is 
much  smaller  than  the  horizontal  estimate.  This  is  the  case  because  the  physical  properties  of 
most  raster  projectors  allow  horizontal  lines  (used  here  for  the  vertical  resolution 
measurements)  to  be  displayed  at  a  greater  contrast  than  the  pixels  within  each  line,  which  form 
the  displayed  vertical  lines. 

Projector  Evaluation  2.  Projector  Evaluation  2  was  perfonned  on  a  second  projector  to 
obtain  additional  data  on  the  variability  of  our  spatial  resolution  measurements.  In  addition,  in 
order  to  assess  possible  interactions  between  test-day  and  test-period,  multiple  measurements 
were  obtained  within  each  test-period.  Shown  in  Figure  3  are  the  average  horizontal  and 
vertical  spatial  resolution  measurements  obtained.  The  error  bars  are  standard  errors  of  the 
three  resolution  estimates  obtained  for  each  test-period. 

Separate  two-way  ANOVAs  were  perfonned  for  the  horizontal  and  vertical  resolution 
data.  For  the  horizontal  estimates,  significant  differences  were  found  for  both  Test-Day  (6*3,24  = 
92 ,p  <  0.001)  and  Test-Period  (.62,24=  375, p  <  0.001),  as  well  as  for  their  interaction  (6*6,24  < 
191,  p  <  0.001).  Likewise,  for  the  vertical  estimates,  significant  differences  were  found  for  both 
Test-Day  (.63,24  =  340,  p  <  0.001)  and  Test-Period  (6*2,24  =  430,  p  <  0.001),  and  for  their 
interaction  (6*6,24  <  264 ,p  <  0.001). 

Because  significant  differences  in  measured  resolution  were  found  for  different  test- 
days  and  different  test-periods,  data  were  not  collapsed  across  either  of  these  variables  in 
obtaining  the  measurement  variability  estimates.  The  ratios  of  the  standard  deviation  to  the 
mean  of  the  36  individual  horizontal  and  vertical  measurements  were  found  to  be  0.077  and 
0.040,  respectively.  The  resulting  variability  estimate  for  our  projector  spatial  resolution 
measurements  was  the  mean  of  those  two  estimates,  0.059  (5.9%). 

Combining  the  variability  estimates  of  Projector  Evaluations  1  and  2  results  in  an 
overall  estimate  of  about  4.3%.  However,  the  variability  estimates  were  much  higher  for  the 
second  evaluation  due  largely  to  one  data  point  obtained  on  the  first  day  of  Projector 
Evaluation  2  (see  Figure  3).  If  that  data  point  is  considered  an  outlier  and  is  removed,  the 
horizontal  variability  estimate  for  Projector  Evaluation  2  is  reduced  to  4.4%,  and  the  overall 
variability  is  reduced  to  3.5%. 

CONCLUSIONS  /  RECOMMENDATIONS 

The  variability  of  spatial  resolution  measurements  obtained  on  CRT  projectors  used  in 
the  M2DART  is  estimated  to  be  about  4.3%.  The  analogous  variability  for  test  stimuli  in  the 
form  of  transparencies  illuminated  by  a  stabilized  light  source  was  about  1.3%.  The  difference 
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in  the  two  variability  estimates,  3.0%,  may  be  taken  as  an  estimate  of  the  variability  of  the  CRT 
projectors  alone. 

For  the  purposes  of  the  present  memorandum,  the  difference  in  the  variability  estimates 
obtained  in  Projector  Evaluations  1  and  2  must  be  assumed  to  be  due  to  inherent  variability  in 
the  output  of  CRT  projectors  of  the  type  evaluated  here.  Given  that  only  two  projectors  were 
evaluated,  however,  it  is  recommended  that  additional  data  be  obtained. 
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Figure  1.  Spatial  resolution  estimates  obtained  for  the  Grating-Transparency  Evaluation 
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Horizontal  Resolution  Summary 
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Figure  2.  Spatial  resolution  estimates  obtained  in  Projector  Evaluation  1. 
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Figure  3.  Spatial  resolution  estimates  obtained  in  Projector  Evaluation  2 
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